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.\BSTRACT 

The External Diploma Program (EDP) of the American 
Council on Education is an applied-performance, competency-based 
assessment program that provides credentials to skilled adults who 
have acquired their high-school-level abilities through their life 
experience. Through the EDP mature adults can do the following: 
demonstrate their abilities in a series of simulations that parallel 
job and life situations; be evaluated against established criteria of 
performance instead of by comparison with others; take responsibility 
for acquiring instruction through existing community resources; and 
demonstrate mastery of all the competencies required, including an 
occupational or specialized skill, without stepping inside a 
classroom. Outcomes were originally determined through a task force 
consensus model, through which a group of busiaess, labor, education, 
and government representatives agreed on life skill competencies. The 
outcomes have been updated periodically as new competencies have 
become necessary. Consistency and stability of the ratings are 
constantly monitored to ensure reliability. The portfolio assessment 
has been kept cost effective because there is no instructional 
component, recordkeeping has been streamlined, new instructional 
programs are not designed, assessments can be delivered at work 
sites, and raters meet frequently to build consensus. The EDP system 
has the following characteristics: (1) documents outcomes; (2) 
includes multiple modes of demonstration; (3) is a high-stakes 
assessment; (4) contains safeguards; (5) is valid; (6) is reliable; 
(7) has quality controls; and (8) can be cost effective. (KC) 
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WHAT IS THE EXTERNAL DIPLOMA PROGRAM (EDP) ASSESSMENT SYSTEM? 

The External Diploma Program (EDP) is an applied-performance, competency-based assessment 
program that credentials skilled adults who have acquired their high school level abilities through 
their life experience. The EDP allows mature adults to demonstrate their abilities in a series of 
simulations that parallel job and life situations; to be evaluated against an er tablished criteria of 
performance instead of by comparison wiUi others; to take responsibility foi acquiring instruction 
tlirough existing community resources; and, finally, to demonstrate mastery of all the competencies 
required, including an occupational or specialized skill, without stepping inside a classroom. 

The skills and competencies demonstrated through the EDP assessment process are those needed 
by a competent citizen functioning in the modem worid. Many of the 65 competencies parallel 
those expected of a skilled employee as determined by the WORKFORCE 2000 and SCANS 
research. Local school boards and state departments of education review the EDP competencies, 
agreeing to award a traditional diploma to adults who complete tiie EDP process with 100% 
mastery. The American Council on Education trains, evaluates and certifies EDP assessors and 
monitors the administration of this nationwide applied-performance assessment process. 

Before an adult can begin documenting her or his learning at die high school level, basic skills in 
reading, writing and math must be demonstrated through diagnostic testing conducted in a series of 
office visits. Specific skills needing remediation are reported to the adult in written and oral 
feedback. EDP has no instructional component; however, adults in need of instruction are referred 
to existing educational resources. Once the basic skill competence is demonstrated, the adult can 
begin the EDP high school level assessment. 

At this stage, the 65 competencies aie embedded in a series of life skill tasks that the candidate 
completes independently, using resources in the community. The candidate then demonstrates 
many of the same competencies again in a second simulation or a structured interview conducted in 
a closed assessment setting. The EDP assessor evaluates the clients' answers according to the 
national criteria contained in the program's manuals. Any item not demonstrated on the first try is 
reported to the candidate, who then seeks help on her or his own and attempts the demonstration 
again. All items demonstrated are compiled in a portfolio of evidence; those not demonstrated 
must be learned and attempted again, until all 65 competencies are demonstt^ted correctiy. At the 
end of the process the adult has compiled a portfolio that represents all 65 competencies which 
have been demonstrated successfully. That final portfolio is completely reviewed by a second 
assessor, according to the manual criteria. 

HISTORY OF EDP, ACE AND OTHER ORGANIZATIONS IN MEASURING 
THE OUTCOMES OF ADULT LEARNING 

ACE, tiirough the Center for Adult Learning and Educational Credentials, has a 50-year 
history of assessing and evaluating the outcomes of learning acquired by adults from a 
variety of educational experiences. ACE recommends to colleges and universities credit 
guidelines for business and industry training courses, learning acquired through the 
military, and credit by examination. GEDTS also asks the adult learner to demonstrate 
competency acquired exti^-institutionally. Besides ACE's newly updated EDP (the "other 
high school assessment option"), other models of alternative assessment have been used in 
large-scale testing situations, ranging from the highly visible certification for professions 
(such as nursing); to the GEDTS essay recently added to die multiple choice Writing Skills 
test; to GAEL models for college portfolios for adults with acquired learning outside of 
school; to Toyota's assessment system for promotion and employment in its Georgetown, 
KY plant 
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EDP: HOW THE OUTCOMES WERE DETERMINED 



The first step in designing any assessment system (or any good instructional system, for that 
matter) is to establish the outcomes to be measured (or taught). Who says what is taught and thus 
what is measured is always an issue, and once again the design of the outcomes traditionally 
follows the purpose of the assessment. If you wish to measure the skills of a plumber or a doctor, 
the field is often asked to set the test specifications, because the outcomes expected of a good 
practitioner are best described by persons specializing in that discipline. If a business or union 
pays for the instruction, they set the learning desired and describe what they want their employees 
to do with the kaming after it has been achieved. These outcomes become both the instructional 
goals and the evaluation criteria. 

For high stakes certification outcomes cannot be decided subjectively by one teacher or 
expert, but rather must be determined through a consensus on the body of knowledge and 
skills that the learner, program, teacher, or world expects of a successful adult functioning 
in that particular task. . 

To establish the original EDP competencies (those necessary for an adult functioning at the level of 
a competent high school graduate in the worid), the original New York designers of the EDP 
alternative assessment system used a task force consensus model with members from business, 
labor, higher education, secondary education, and government. In 1973, this group generated a 
list of more than 100 generalized competencies of life skills needed by competent adults 
(functioning at the level of a high school graduate) to cope in society. 

A group of 38 educators then rated the 100 life-skill competencies to prioritize, level, and clarify 
them. Some competencies were found to be repetitive, nonessential, or more sophisticated than the 
skill level required of a high school graduate functioning successfully in society. Those 
competencies were eliminated and the group produced a final list of 64 life-skill competencies 
within seven generic theme areas. 

It was also determined that each adult would be required to demoriStrate competence in one of 
three individual areas: 

• Skill in an occupational or vocational ai*ea, OR 

• Skill in an advanced academic area, OR 

• Skill in a specialized area, such as art or home management. 

In 1991 ACE reviewed and revised the original competencies using a consensus group involving 
representative employers, union leaders, community leaders, adult teachers, and secondary school 
principals. The suggested list was compared with the WORKFORCE 2000 competencies, 
speculating about what a competent worker of the 21st century will need. Academic subject matter 
experts (writing, reading, math) and life skill experts (American Red Cross, American Bar 
Association, USDA, Planned Parenthood, and others) reviewed the materials. Before each new 
implementation of the EDP, the new 65 EDP competencies are reviewed by local school boards. 
To date, more than 40 school boards have reviewed the competencies and agreed to award the 
diploma for demonstration of these competencies. The competencies have been further reviewed 
and accepted as more than sufficient by several EDP partners representing large companies, labor 
unions, and small business organizations. 

To document the learning claimed by the student, EDP includes a variety of non-traditional modes 
of demonstration: oral, written, applied performance, documentation, and simulations. The EDP 
demands a 100% mastery of all competencies, but allows for the adult to self-pace the learning and 
the demonstration of that learning. Parts of the assessment (called "tasks") are done at home and 
parts (called "spot check/interviews") are assessed in a series of private office visits. The criteria 

2 



ERLC 



4 



both of what is acceptable evidence and the level of skill demanded are outlined in standardized 
manuals and in the directions to the candidate. The burden for compiling most of the evidence is 
placed on the candidate. 

EDP: HOW THE SAFEGUARDS NECESSARY FOR HIGH STAKES TESTING 
WERE ACHIEVED 

To insure reliability of the alternative assessment in any high stakes situation demands clearly 
stated criteria that can be administered objectively and universally. In a high stakes certification 
process sucn as one that issues a high school diploma, vigilance is needed to maintain a 
consistency and objectivity in evaluation among raters. In the case of the EDP, portfolios are first 
reviewed against nationally established criteria, by raters known as assessors who are trained and 
certified by ACE. Each assessor's evaluation of the complete demonstration of the 65 
competencies is completely evaluated a second time by another assessor before the diploma is 
awarded. ACE has built in procedures for assessors to monitor their own evaluations to insure 
consistency at each local program level. At the end of the first year of implementation and 
periodically thereafter ACE conducts an inter-rater reliability check, with feedback, to monitor 
state-wide and nation-wide consistency and stability among raters, 

EDP: ACHIEVING COST EFFECTIVENESS 

Cost effectiveness in all portfolio assessment and alternative assessment processes is always a 
concern. EDP was designed to combine the highest level of assessment reliability with the most 
efficient use of staff time. This v/as achieved in several ways: 

1. EDP is strictly an assessment system; there is no instructional component. Since the major 
burden of learning is placed on the student and achieved outside the teacher's contact hours, the 
agency pays only for staff time used for administering and evaluating the assessment, not for 
learning time. Approximately 30 hours of staff time per graduate is required for administering and 
evaluating the assessment once a staff person completes the first cycle of training. At $20 per hour 
of staff time, this makes staffing cost per graduate approximately $600.00. In addition, materials 
per graduate cost $50. Since the average grade completed for most EDP grads is 10th grade, the 
same graduate attending class in an adult high school would take an estimated 10 more classes to 
graduate and would cost approximately $1080.00 a grad (with 20 persons in a class, at 
approximately 108 hours per class, teacher paid $20 an hour). 

2. The recordkeeping and feedback sheets (often a high cost item in a self-paced, competency 
based model) is streamlined by means of checklists. 

3. No neexi exists for new instructional programs to be designed. New clients are led to existing 
programs, including community services such as libraries, book stores, and television. 

4. Partnerships can be facilitated as assessment can easily be delivered at members' sites, with part 
of the staffing and material costs picked up by businesses, labor, PICS, and other partners. 
Staffing cati be part-time houriy. 

5. Inter-rater reliability is designed as an integral part of the local site*s procedures: 

• Training and evaluation are combined in the same up-front costs; the evaluation 

workshop is designed to turn the monitoring over to the local site director. 

• Portfolio reviewers (every portfolio reviewed by a second assessor) traintxl to become 

the "eyes of ACE." 

• Consensus moderation (achieved by staff meetings) is built-in to achieve a continuous 

training model. 
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SELECT CHARACTERISTICS OF THE EXTERNAL DIPLOMA PROGRAM 
(EDP) AS AN ASSESSMENT SYSTEM 



EDP DOCUMENTS OUTCOMES RATHER THAN INPUT 

• focused on what is learned rather than what is taught 

• documented according to standard criteria shared with the learner 

• evaluated according to standard criteria (criterion-referenced) 

• includes credit for prior learning 

EDP INCLUDES MULTIPLE MODES OF DEMONSTRATION: observations/ written and oral 
demonstrations/ structured interviews/ simulations/ projects/ documentation 

• demonstration mode appropriate to the specific skill being measured 

• inclusion of various learning style strengths 

• specific feedback and resource information shared with the learner 

• unlimited opportunities to demonstrate skills 

• no time barriers, unless relevant to demonstration of specific skill 

• many opportunities for the adult to present her or his own situational examples 

EDP IS A HIGH STAKES ASSESSMENT 

A high stakes assessment is any assessment or testing system that selects or screens persons by means of 
this test or demands accountability from programs awarded funding 

• educational credentials 

• license 

• personnel tests: hiring or promotion 

• professional certification 

• vocational certification 

• accountability 

WHAT ARE NECESSARY SAFEGUARDS FOR ASSESSMENT IN A HIGH-STAKES SITUATION? 

• Validity of the measures 

• Reliability among raters 

• Same opportunity for all (no bias to race, gender, ethnicity, etc.) 

• Reliability and transferability of the results 

EDP IS VAUD 

• reflects the reality of the job, some external standard, or a definition of what the 
outcomes are expected to describe 

• inclu-ies relevant and significant abilities an adult is expected to demonstrate under 
the above 

• no bias to race, gender, ethnicity, etc. 
HOW TO ACHIEVE FI: 

• deteiTnined through a consensus process of users or practitioners 

• agreement to standards by users or practitioners 

4 



ERLC 



EDP IS RELIABLE 



• raters use the same standards and criteria for acceptable pcrfom>ance 

• raters consistently interpret the criteria in the same way 

• in judgmental calls there is a consistent process for reaching consensus 

• raters stay reliable over time and administrations 

HOW TO ACHIEVE IT: 

• consistent staff training and follow-up training opportunities 

• monitoring of assessment procedures by an organization of repute concerned with 
quality 

» restriction on the use of assessment instruments to trained staff 

• no permission to alter instruments 

• consensus moderation 

• inter-rater reliability monitoring over time 

EDP HAS QUALITY C0^4TR0LS 

• reliable agency 

• monitoring and systematic checks on raters 

• standardized and unbiased procedures for administration, scoring, and giving 
feedback 

HOW TO ACHIEVE IT: 

• local, as well as national monitoring 

• on-going training and training the trainers 

• double or cross checks among assessors 

EDP: APPUED PERFORMANCE/PORTFOLIO ASSESSMENT CAN BE COST EFFECTIVE 

EDP is more cost effective than "seat-time options/' since the average grade completed 
for most EDP clients is 10th grade. Qasses requiring 108 hours X 10 classes/ 18 students 
per class (60 hours per grad X teacher's salary $20= $1200.00 per grad.) EDP (same 
salary of $20 for assessor) X 30 hours staff time per graduate=$6()0.00 per grad. 

HOW TO ACHIEVE IT: 

• high performance staffing roles, no down time; assessors time always productive: 
appointment part independent, part interview, part time for evaluating tasks; trained 
for all roles. 

• short-cuts for recordkeeping: using self-assessing and checklists to report and to keep 
records. 

• more time spent on learning by the candidate than the teacher; system only pays for 
contact hours with the staff 

• uses existing learning opportunities, including training 

• goal of final training and evaluation is for self-sufficiency and to train sites to 
monitor themselves; sporadic checks done by mail. 
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QUESTIONS FOR REVIEWING A S SESSMENTS 

IS MY ASSESSMENT fflGH STAKES? 
does it issue an educational credential? 

does it determine who goes on to college or to further educational opportunities? 

^ does it issue an license? 

is it used for hiring or promotion? 

does it issue a professional certification? 

does it issue a vocational certification? 

does it effect my funding? 

SAFEGUARDS FOR THE ASSESSMENT : 

Consistent staff training? 

Follow-up tiaining opportunities? 

Same opportunity for all (no bias to race, gender, ethnicity, etc)? 

Reliability among raters? 

Inter-rater reliability monitoring? 

Restriction on the use of assessment instruments to trained staff? 

No permission to alter instruments? 

Reliability and transferability of the results? 

Monitoring of assessment procedures by an organization of repute concerned with 

quality? 

IS THE ASSESSMENT VALID? 

outcomes determined through a consensus process of users or practitioners 

agreement to standards or outcomes by users or practitioners 

reflects some external standard or a definition of what the outcomes are 

no bias to race, gender, ethnicity, etc. 



IS THE ASSESSMENT RELIABLE? 

raters use the same standards and criteria for acceptable performance? 

raters consistently interpret the criteria in the same way? 

in judgmental calls is there a consistent process for reaching consensus? 

do raters stay reliable over time and administrations? 

ARE THERE QUALITY CONTROLS ? 

reliable agency 

monitoring and occasional checks on raters 

standard systematic and unbiased procedures for administration, scoring, and 

giving feedback 

HOW CAN YOU MAKE THE ASSESSMENT SYSTEM MORE COST EFFECTIVE? 

Time and duties of staff (high performance staff ing roles) 

Recordkeeping 

Training for self-sufficiency 

J Learner does more work than the teacher/assessor 

Use external existing opportunities (don't re-invent the wheel) 
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